Maximum Entropy Models and Stochastic Optimality Theory
نویسنده
چکیده
In a series of recent publications (most notably Boersma (1998); see also Boersma and Hayes (2001)), Paul Boersma has developed a stochastic generalization of standard Optimality Theory in the sense of Prince and Smolensky (1993). While a classical OT grammar maps a set of candidates to its optimal element (or elements), in Boersma’s Stochastic Optimality Theory (StOT for short) a grammar defines a probability distribution over such a set. Boersma also developed a natural learning algorithm, the Gradual Learning Algorithm (GLA) that induces a StOT grammar from a corpus. StOT is able to cope with natural language phenomena like ambiguity, optionality, and gradient grammaticality, that are notoriously problematic for standard OT. Keller and Asudeh (2002) raise several criticisms against StOT in general and the GLA in particular. Partially as a reaction to that, Goldwater and Johnson (2003) point out that maximum entropy (ME) models, that are widely used in computational linguistics, might be an alternative to StOT. ME models are similar enough to StOT to make it possible that empirical results reached in the former model can be transferred to the latter, and these models have arguably better formal properties than StOT. On the other hand, the GLA has a higher cognitive plausibility (as can be seen from Boersma and Levelt (2000)) than the standard learning algorithms for ME models. In this paper I will argue that it is possible to combine the advantages of StOT with the ME model. It can be shown that the GLA can be adapted to ME models almost without modifications. Put differently, it turns out that the GLA is the single most natural on-line learning algorithm for ME models. Keller and Asudeh’s criticism, to the degree that it is justified, does not apply to the combination of ME evaluation with GLA learning, and the cognitive advantages of the GLA are maintained.
منابع مشابه
A Robbins-Monro type learning algorithm for an entropy maximizing version of stochastic Optimality Theory
The object of the present work is the analysis of the convergence behaviour of a learning algorithm for grammars belonging to a special version the maximum entropy version of stochastic Optimality Theory. Stochastic Optimality Theory is like its deterministic predecessor, namely Optimality Theory as introduced by Prince and Smolensky, in that both are theories of universal grammar in the se...
متن کاملControl Theory and Economic Policy Optimization: The Origin, Achievements and the Fading Optimism from a Historical Standpoint
Economists were interested in economic stabilization policies as early as the 1930’s but the formal applications of stability theory from the classical control theory to economic analysis appeared in the early 1950’s when a number of control engineers actively collaborated with economists on economic stability and feedback mechanisms. The theory of optimal control resulting from the contributio...
متن کاملOptimality Of Monetary And Fiscal Policies In Iran: An Application Of The Stochastic Optimal Control Theory
متن کامل
A general necessary and sufficient optimality conditions for singular control problems
We consider a stochastic control problem where the set of strict (classical) controls is not necessarily convex and the the variable control has two components, the first being absolutely continuous and the second singular. The system is governed by a nonlinear stochastic differential equation, in which the absolutely continuous component of the control enters both the drift and the diffusion c...
متن کاملNecessary and sufficient optimality conditions for relaxed and strict control problems of forward-backward systems
We consider a stochastic control problem of nonlinear forward-backward systems, where the set of strict (classical) controls need not be convex and the coefficients depend explicitly on the variable control. By introducing a new approach, we establish necessary as well as sufficient conditions of optimality, in the form of global stochastic maximum principle, for two models. The first concerns ...
متن کامل